Scalable and Non-Intrusive Load Sharing in Owner-Based Distributed Systems

Figure 10: Intrusiveness on the LANs for the distributed algorithm with a heterogeneous load. 7 Conclusions We have shown that existing approaches to load sharing do not scale while supporting a rich set of policies and satisfying bounds on intrusiveness. Further, we have argued that in an owner-based distributed system , it is not sucient to choose a load sharing algorithm which typically induces little overhead on the nodes and LANs; rather, the load sharing algorithm must be able to regulate and control the overhead. Finally , we outline the distributed clustering algorithm that provides scalable and non-intrusive load sharing for a rich set of sharing policies. [10] H.-C. Lin and C.S. Raghavendra. A dynamic load-balancing policy with a central job dis-patcher. the centralized algorithm with a heterogeneous load. Figure 6: Intrusiveness on managers for the DC algorithm with a heterogeneous load. Figure 7: Intrusiveness on nodes for the distributed algorithm with a heterogeneous load. Figure 8: Intrusiveness on the LAN of the manager for the centralized algorithm with a heterogeneous load. DC algorithm with a heterogeneous load. Figure 2: Intrusiveness on the nodes. distributed algorithms introduce very little overhead. We now consider a heterogeneous environment in which location policy requires us to rst search for an available processor within the local cluster and then to try all the processors in each neighboring cluster. Cluster size is xed at 20 processors per cluster. The simulated receiving policies are such that half of the clusters will not accept any remote tasks, though they are still generating remote task requests at an average rate of one request per minute per node. The other half of the clusters are relatively idle, but their nodes are generating state update messages at the rate of one update per 10 seconds per node. The results (see Figure 4) show that both our DC approach and fully distributed load sharing scale beyond 1400 nodes; however , response times are an order of magnitude higher in the distributed case. For this workload, the centralized case fails to scale beyond about 1000 nodes. The tables in Figures 5{7 report the fraction of compute cycles expended on load sharing overhead. The latter two tables report overhead distributions rather than simple averages, with each column representing an intrusiveness range. For example, in Figure 6, we see that for a 1400 node system, four cluster managers spent 11{13.5% of their cycles on …